docs(databricks): amend-and-build runs the dbt job and waits for it (forge-cli#861) - #207

What

Documents forge-cli #861: on Databricks, fluid apply --mode amend-and-build and --mode replace-and-build now start a run of a dbt build's provisioned job and wait for it, and FLUID_DATABRICKS_RUN_TIMEOUT_SECONDS caps the wait.

• providers/databricks.md: a new section, Running the job from fluid apply. It covers:
  • which modes start a run, --build-id, continuous jobs;
  • how the job is found (state tags, never the name), and that the run writes nothing to the job (schedule and pause state kept, --full-refresh for that run only);
  • the queue and the idempotent start;
  • the output, with live captures of a success and a failure, and the two queued reasons;
  • the wait cap, with a worked 3 h 25 min example; cancellation at the cap, on Ctrl-C, SIGTERM and SIGHUP, and nohup; a run that cannot be read;
  • the missing-SDK refusal and --dry-run;
  • building an estate in lineage order.
• Also on that page:
  • the overview bullet;
  • the Prerequisites row for the [databricks] extra;
  • the paused-job note;
  • the manual trigger row;
  • "Applying and removing";
  • "How it is tested" (the 2026-11-12 live run, and what was not seen live).
• Other pages:
  • providers/README.md: the Databricks summary cell.
  • cli/apply.md: a build-phase paragraph and the apply_databricks_sdk_missing row.
  • advanced/environment-variables.md: the new variable.
  • advanced/error-codes.md: the code is not catalogued.
  • advanced/operating-in-ci.md: the CI job waits, and a cancel cancels the run.
  • upgrading.md: a "Not in a release yet" entry.

Fixed in passing: the Prerequisites row said "Only fluid verify needs" the [databricks] extra. Since 0.27.0, fluid apply needs it as well, to bind column masks and row filters and to land a DuckDB acquisition. The page's own masks section already said so for masks.

Markers

Every new sentence carries the branch form, because #861 is not merged: On forge-cli's feat/databricks-build-run-now branch (forge-cli #861), not yet on main or in a release.

• Checked against: #861 head cd65af29. Later heads change nothing this PR documents:
  • a96a3636 changes one comment and one test.
  • 77715548 merges main (forge-cli #860 Unity Catalog keys, #862). It routes the build phase through build_after_apply, which keeps the exit code.
  • #860's optional fluid_keys task adds its own timeout to the wait cap, as the per-task formula says.
• When #861 merges: switch every marker to the main form, and keep the "Not in a release yet" heading in upgrading.md.
• Draft until then: the text says what the branch does, so it should not go live before #861 lands.

Output samples

• Success, failure, dry run, missing SDK: captured live on Databricks Free Edition on 2026-11-12, in a scratch catalog that was dropped afterwards. The workspace host and id are replaced.
• The failure block: abridged. Its task line is shortened with …, and the wrapped error line is joined.
• The two QUEUED - ... reason lines: taken from the code. They were not seen live, and the page says so.

Checks

On the pinned CLI 0.29.0, Python 3.11:

• scripts/check_cli_docs.py, also with --version-only and --markers-only (with FORGE_CLI_CHANGELOG set to forge-cli main's CHANGELOG): OK.
• scripts/check_providers.py: OK.
• scripts/gen_contract_reference.py --check: OK.
• npm ci && npm run docs:build && node scripts/check-dist-links.mjs: build OK, links clean. Every new #anchor target exists in the built providers/databricks.html.
• A verifier agent checked every claim against #861's head, and the 0.29.0 sentences against v0.29.0.
  • It found no wrong claim and six imprecise ones. All six are fixed:
    • a failed cancel prints the run page only at the cap;
    • after five failed reads, the page is printed only when an earlier read returned it;
    • a timeout of more than nine digits is not used either;
    • a job created paused is unpaused by the second apply;
    • the SDK gate skips --dry-run and continuous builds;
    • a queued line has no reason when the job's runs cannot be listed.
  • It reproduced apply_databricks_sdk_missing for real, and the dry-run lines and the 12300 s cap in code.
  • Its runs of the head's two job-run test files passed (73 tests).

Not in this PR

• forge-cli's fluid doctor still says databricks-sdk is "needed by fluid verify only" (fluid_build/cli/doctor.py). That has not been true since 0.27.0, and #861 adds the build phase. It is a forge-cli fix.
• cli/apply.md's "Which engine runs" table lists aws, gcp, snowflake and confluent for OpenTofu, not databricks.